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DESCRIPTION 



GRAPHIC IMAGE MESSAGE GENERATION 

5 The present invention relates to the formation and transmission of 

graphic images and particularly, but not exclusively, to the sending of graphic 
objects as all or part of a message between users of hand-held or mobile 
communications devices. 

10 The majority of the current generation of hand-held mobile devices are 

driven by text-based user interfaces and deliver text-only messages in paging 
and similar operations. Through these interfaces, the user is enabled to browse 
the handsets range of functionality and select and perfomi the required tasks. 
Such mobile devices can also exchange information using protocols such as the 

15 Short Message Service (SMS) part of the Global System for Mobile 
communications (GSM) digital standard. These text based displays have . a 
limited capability and lack the flexibility of display and appeal of graphical 
infonnation. Increasingly there is a desire to incorporate graphical information to 
improve both the user interface and the exchange of information. 

20 An example of a mobile messaging system seeking to improve the ready 

understandability of messages through the use of displayed graphic icons to 
represent or support a message is described in International patent application 
W097/19429 (Motorola/Deluca et al). In the mobile telephone receiver 
described, a graphics database holds image data for a number of 

25 ■ predetermined images, each identified by a respective (and standardised) code. 
On receipt of a paging message from another user, the handset processor 
scans the message firstly to identify whether it contains the code or codes for 
one or two of the images stored in the graphics database and, on finding these, 
it generates these images on a display screen of the handset. Secondly, it 
30 scans the message for any character string data which, if found, will be 
converted to a text message and displayed on screen below the image or 
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images. 

Whilst this arrangement can add to the appeal and understandability of 
the paging message, its flexibility is limited by the need for the image data to be 
pre-loaded to the handset: although various techniques are described for 
5 updating the handset's graphic database, from manual data entry through to 
over-the-air downloading of graphics data files, the problem remains that the 
identity of each of this finite set of images must be standardised - at the very 
least between pairs of users who intend to utilise the service. 

10 It is. therefore, an object of the present invention to provide improved 

flexibility in the provision of graphics images to be displayed on the screens of 
hand-held or mobile apparatus, which images may represent the whole or part 
of a message. 

It is a further object to provide a. communications apparatus configured to . 
15 generate such graphic images. 

In accordance with a first aspect of the present invention there is 
provided a method for the generation of paging messages for passing between 
remote data communications apparatuses, wherein the message generating 
apparatus constructs a coded message Including specification of one or more 
20 graphic message elements and transmits the coded message via a server to the 
, message . receiving apparatus which decodes and displays the messiage; . .. 

characterised in that the message generating apparatus constructs the 
graphic message elements from one or more locally stored image component 
objects, codes the message as an ordered list of such component objects and 
' 25 their relative positions within an image field which image field . is recreated based ^ 
on stored protocols for handling such objects within the message receiving 
apparatus. By making use of an internal store of component objects from which 
a customised image may be constructed, a much greater range of possible 
graphic messages is possible and, with the receiver apparatus handling the 
30 relatively simple protocols for recreating objects (as opposed to whole images) 
neither the processing load at the receiver, nor the bandwidth for image data, 
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are excessive. 



In order to expand the devices component store, tlie said server may 
maintain a database of image component objects, with the message generating 
5 apparatus being enabled to access said database, download via 
communications link data defining a selected one or more image component 
objects, and store the same. The selection and downloading may be wholly or 
partly under the direction of the user of the sender device, with the server 
perhaps presenting menus of available objects. To avoid the need for further 
10 telecommunications protocol specification to support the service, 
communications between said apparatuses and via the server may suitably 
confonn to GSM protocols with the sending of coded graphic data utilising the 
abovermentioned SMS function of GSM: it is a particular feature that the graphic 
message data should be extremely compact, as will be discussed hereinafter. 
15 . . Also in accordance with the present invention there is provided a portable 
communications apparatus for use as a message generating apparatus in the 
above-recited method, the apparatus comprising; a first mernory. cornprising a.. 
. store of component objects; a display device controllably operable to display 
one. or more component .objects recalled from the first memory: user operable- 
20 control means for selecting and manipulating displayed component objects; 

encoding means for generating a: coded specification of said..seLe.cted.,.and. 
^ manipulated component objects; and transmitter means for formatting and 
sending, via a server, a paging message including, said coded specificatlpn. 

The encoding means may be arranged to generate an ordered rendering, 
25 list identifying, the order in which the selected .component objects .are to be 
rendered on regeneration of the image, and to sequentially code each object of 
the list as a first data word identifying the object type and one or more further 
words specifying coordinates for said object. The apparatus may further 
comprise a receiver coupled with the said first memory and configured to 
30 receive coded image data from the server, and further storage means coupled 
with the display and containing said object identifier codes, the display being 
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configured to reconstruct the image from the object data and output the same. 

The control means may comprise a touch screen overlaying the display, 
in which arrangement the control means may suitably be configured to monitor 
and store component objects specified by user input in the form of entry via the 

5 touch screen. With this feature, the freehand drawing of component objects by 
the user may be supported, such as to enhance the personalised nature of 
messages and increase the variety of possible graphic images. - Alternately, or 
additionally, where the device has a number of existing user operable controls, 
the said input means may be provided as selectable secondary functions of one 

10 or more of those user operable controls. 

Thus the present invention provides -a- means- for~exchanging graphical 
images for display in a format optimised for low bit-rate mobile data 
communications. The method allows objects to be displayed in a device 
independent manner on low to medium resolution graphical displays such as 

15 those of mobile telephones or PDA's. 

- Preferred embodiments of the present invention 
by way of . example only and with reference to the accompanying .drawings.jn ... 

' r whichc.-- ' - . —r- ^ - • ^> - .v:._....-:: -.: r.-^ 

20 Figure 1 is a block schematic diagram of a messaging, systern embodying 

jthe present invention; - . .\ 

Figure 2 is an example component object image formed from, seven _ 

- -- - objects; - - / -- --^ < i,^- r --r-;c^o- . .^: = :i i.^ : V. -i-i: - 

Figure 3 is a table of the respective properties pf^each of the objects. in 
. . . 25 the. image of Figure 1; " -r^^ . - .c _ . . 

: Figure 4 represents the bit-structure, of a cpmnriand word, identifying _an _ 

object; 

Figure 5 represents the bit-structure of an object coordinate specification; 

and 

30 Figure 6 is a block schematic diagram of a receiver apparatus suitable to 

embody the present invention. 
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Figure 1 generally illustrates a messaging system which may suitably 
embody the present invention, comprising a number of client personal 
communications devices 100 exchanging messages using the SMS part of the 
5 GSM digital standard via a sen/er 200. If a client is unreachable, then a 
message will be stored at the sen/er 200 and sent later as the target client 
becomes available. The present graphic messaging system sits on top of the 
existing SMS infrastructure; SMS allows short textual messages to be sent 
between telecommunications devices, with messages being entered by typing 
10 on the devices keypad, and the applicants have recognised that this may be 
extended-to support the sending of suitably-formatted graphic image data. 

In general operation, a client device 100 provides the user with means in 
the form of a simple graphics editor to assemble graphic images from one or 
rmore component objects (which objects may include text strings), as will be 
15 described, with the device maintaining an internal store 150 of such component 
objects. Preferably, in a basic mobile telephone application, all interactions or 
object manipulations specified by the user are pferfomned by the standard 
- telephone keys:-fof example the 4 and 6 humeric keys to 

move a selected object left or right, and the 8 and 2 keys to move up or.down. 
20 In order to enhance the number of component objects available to a user 

composing an image, the server 200 niay maintain or provide direct access to a 
library 250 of additional objects.- Using either SMS or a GSM data channel, the 
user can browse a hierarchical-menu. of images and select one for downloading 
to the users device. Alternatively, or additionally, by use of the same numeric 
.25 keys for object manipulation, the user may. be enabled to "draw" freehand. In a 
further extension to this, where -the user device features a touch-screen user 
input, direct drawing of component objects by use of a stylus and the screen 
may be accommodated. 

In order to permit the direct transmission of graphic images to the client 
30 devices, a component object specification is used which describes an image by 
the objects comprising it (lines, polygons, points/circles etc.): a particularly 
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6 

suited coding specification is described briefly below and in greater detail in the 
commonly-assigned United Kingdom patent application 9800900.4 entitled 
"Graphic Image Generation and Coding", filed 17^ January 1998. 

Component object images as used in the following examples are 

5 comprised of three main drawing objects: points, text, and polygons; a polygon 
may have between one and nine edges and, assuming a monochrome or two- 
colour display, may -be filled or unfilled. Each object can be drawn in either 
"black" (with pixels set) or white (pixels are cleared). Figure 2 shows an 
example component object image 10 constructed from a total of seven objects 

10 11-17. Note that it is possible to overlay objects on top of each other; objects 
are drawn in the order in which they are specified in the encoded data stream 
and the encoded data therefore represents an ordered list of display objects. 
The properties of each object in the example image of Figure 2 are as shown in 
the table of Figure 3. Because object 14 (a white filled polygon) was drawn after 

15 object 13 (black filled polygon) it appears on top of the arrow-shaped polygon 
1 3. Likewise the text string (object 15) appears on top of the white filled polygon 

All vertices in component object images are specified in a fixed 
coordinate image field using the XY coordinate system shown in Figure 1, with 

20 the origin in the top left hand corner and a range 0-127 for each axis. Although 
the image field is "square" (the ranges for the X and Y axes are the same), the 
display device is unlikely to be. The image will be scaled in the terminal to fit the 
display size ahd aspect ratio. . - 

The encoding of component object.data Is intended to be particularly 

25 compact: every object to be drawn is sent a command word followed by a 
variable length of data. The data may be characters (7-bit ASCII) or coordinates 
(14-bits). The commands are formatted as 7-bit words of data. The overall bit 
format is as shown in Figure 4, with all data being encoded in the order of most- 
significant to least significant bit. The first bit, c, is the colour bit and indicates 

30 whether the object perimeter line is to be drawn in the background colour 
("white"; c=0) or the foreground colour ("black"; c=1). The next two bits. CC, 
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indicate the type of object (point, text, filled or unfilled polygon), following which 
is one bit. s, indicating whether or not object may be selected at the device of a 
receiving user (0=no; 1=yes), then three bits, ppp, to specify a parameter value 
of 0-7. Commands may be followed by coordinates in terms of the image field 
5 and formatted as shown in Figure 5. with the first seven bits giving a value 0-127 
for the X coordinate, and the other seven bits giving a value 0-127 for the Y 
coordinate. 

A block schematic diagram of a mobile/hand-held communications 
device configured both to receive and generate component object encoded 
10 messages is shown in Figure 6. The functioning of many of the component 
- pirfs wnrBei(7girunderstood by the skilled reader and. other than In terms of " 
their bearing on the operation of the present invention, they will not be described 
in detail. ' 

From ah aeriaf 20 of other Vignal input, a received signals passed via 
15 transceiver 22 and decoder 24 stages to a central processing unit 26, which unit 
also (in this embodiment) handles the function of display driver formatting 
graphical and text data -into displiay comnVahds for output to display, screen 28. • 
- Ah apparatus merhbry^^ 3^^ doupled- with the processor and" holds basic 
' operating data and programs for- ttie device. User-input to the processop;?26 is-. 
20 via controls 32: these may-be in the form of a simple keypad and/or they may be 
■ - combined with the display. 28 through a touch-screen arrangenrient; as indicated 
by dashed lln^ 34. Other conventional functions of the device (for example 
- audio handling) are illustratedfgenerally at- 36. - • . 

■ ■ Coupled with the processor 26 is a further store 38 containing a collection 
25" of component objects as well- as the command word codes (referred to above) " 
to enable the processor/display driver 26 to Interpret received command words 
and recreate the encoded image, this memory may also hold a table of ASCII 
codes for text strings. Also coupled with the processor 26 is a scaling stage 40 
which may comprise a simple hardware arrangement of multipliers to convert 
30 the image field coordinates in a received data stream to fit the available area 
and aspect ratio of the display device 28. Finally, output signals from the 
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processor 26 such as speech messages, indications of object selections, 
access messages to the server library of additional component objects, or new 
graphic images generated on the device as described previously are sent via 
encoder 42 to the transceiver 22 for sending to the server (200; Fig.1). 
5 As will be recognised, many variations are possible on the arrangement 

of Figure 6. For example, the object and command word code store 38 may be 
incorporated with the general device memory 30. Also, the decoder 24/encoder 
42 and/or scaling circuit 40 functions may be embodied completely in software 
and carried out in processor 26, Alternatively or additionally, the display driver 
10 functions of the processor may be handled by a separate unit, optionally with a 
display memory from which display data would be read out. 

From the foregoing, it will be seen that we have provided a mechanism 
whei-eby objects may be coded in a format optimised for low bit-rate mobile data 
communications. The coding scheme allows .objects:^- be mapped to relative - 
15 screen coordinates enabling the images to be rendered in a device-independent- 
manner: Additionally, since the objects are given a separate, identity in the 
coding scheme, they may also be separately highlighted and selected on the 
screen. Information regarding which object is selected may. be communicated 
back'to the originator enabling navigation and interactive displays-;^ ' : z: : L ^ 
20 From reading the present disclosure, other modifications will be apparent 

to persons-skilled in the art. Such modifications may . involve other features 
J, " which are already known in the design, manufacture and . use ofiJmage coding v:: 
- : equipment and component parts thereof and which may:be usied instead of or in: 
addition to features already described herein. Although claims have been / 
-25 formulated in this application to particular combinations of features^ it should be 
understood that the scope of the disclosure of the present invention also 
includes any novel feature or any novel combination of features disclosed herein 
either explicitly or implicitly or any generalisation thereof, whether or not it 
relates to the same invention as presently claimed in any claim and whether or 
30 not it mitigates any or all of the same technical problems as does the present 
invention. The applicants hereby give notice that new claims may be 



PCT/1B99/00036 



WO 99/37105 



9 



formulated to such features and/or combinations of features during the 
prosecution of the present application or of any further application derived 
therefrom. 
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CLAIMS 

1. A method for the generation of paging nnessages for passing 
between remote data communications apparatuses, wherein the message 
5 generating apparatus constructs a coded message including specification of one 
or more graphic message elements and transmits the coded message via a 
server to the message receiving apparatus which decodes and displays the 
message; 

characterised in that the message generating apparatus constructs the 
10 graphic message elements from one or more locally stored image component 
objects, codes the message as an ordered list of such component objects and - 
their relative positions within an image field which image field is recreated based 
on stored protocols for handling such, objects. wAhm the rnessage receiving 
• apparatus. :r~ . >r . ;^ ' . : " r . ■ v - ~ - ^ ' 

2,. - A method as claimed in Claim 1, wherein., the said, sp^^ 
, . , . ^ . ... maintains- ;..a database. ;.of image cpmponent objects , .and . Xhe ^ message, 
generating apparatus is. enabled to access said database, download .vig . . 
. communications -link . data, defining^^a selected- one or more, image . cpmppoent 
20 objects, and store the same. ' . : . 

. .. . 3. A .method a^ 
V" . .betweeo.said apparatuses and via ihe server .^^^^^ to G3M protQcpIs and^^^^ 
. sending of coded graphic data utilises the SMS. function of GSM. ^; v-^ ; - 

- - - 4. A portable communications apparatus for use as a message 

generating apparatus in the method of Claim 1 , the apparatus comprising: 
a first memory comprising a store of component objects; 
a display device controllably operable to display one or more component 
30 objects recalled from the first memory; 

user operable control means for selecting and manipulating displayed 
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component objects; 

encoding means for generating a coded specification of said selected 
and manipulated component objects; and 

transmitter means for formatting and sending, via a sen/er, a paging 
5 message including said coded specification. 

5. Apparatus as claimed in Claim 4, wherein said encoding means is 
arranged to generate an ordered rendering list identifying the order in which the 
selected component objects are to be rendered on regeneration of the image, 
10 and to sequentially code each object of the list as a first data word identifying 
the object type and one or more further words specifying coordinates for said 
object. 

6 Apparatus as claimed in Claim 5. further conhprising a receiver 
15 coupled With said first memory and configured to receive coded image data from 
the server, and further storage means coupled with said display and containing 
said object identifier codes, said display being configured to reconstruct the 
image from the object data and output the same. ' 

20 7. Apparatus as claimed in Claim 4, wherein said control means 

comprises a touch screen overlaying said display. 

8. Apparatus as claimed in Claim 7, wherein said control means is 
configured to monitor and store component objects specified by user input in the 

25 form of entry via the touch screen. . 

9. Apparatus as claimed in Claim 4, further comprising a plurality of 
user operable controls wherein said input mieans are provided as selectable 
secondary functions of said user operable controls. 

30 
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Obiect 


Type 


11 


Black unfilled polygon, with 1 edge. 


12 


Black unfilled polygon, 4 edges. 


13 


Black filled polygon, 7 edges. 


14 


White filled polygon, 4 edges. 


15 


Black text string. 


•16— 


- Black point, radius 1 — • — - -• - 


17 


Black point, radius 4. 



bit 13 



bit 6 



FIG. 3 



FIG. 4 



FIG. 5 
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